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Abstract: Since the mid-1990s, there has been an abrupt growth of doctoral enrollments 
and doctoral programs in Argentina and Latin America. However, completion rates at this 
education level are far from satisfying. Attrition rates in Social Sciences doctoral programs 
“are known” to be high, and higher in Social Sciences and Humanities (which will be later 
called “Soft Sciences”) than in STEM (which will be later called “Hard Sciences”) 
disciplines, although there are no valid and reliable data. In order to address this problem, 
we carried out an investigation with two objectives: (1) to measure performance (efficiency 
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or productivity) in 18 doctoral programs in STEM disciplines, the Social Sciences, and 
Humanities at various universities in Buenos Aires city and the surrounding Metropolitan 
Area; and (2) to design a procedure for collecting valid and reliable information that would 
allow us to assess doctoral programs' performance. This measure was defined along two 
dimensions: completion rates (regular time rates and extra time rates) and average time-to- 
degree by cohort. We compiled data on individuals' academic tracks in each cohort from 
their enrollment to their completion or dropout by using paper-files and digitized 
databases provided by academic departments. Information was analyzed by program, by 
cohort, and by six grouped cohorts for 2001-2006. Substantive and methodological results 
were found. Among the substantive results, we discovered better performance in STEM 
disciplines than in the Social Sciences and Humanities. As to the methodological results, 
postgraduate statistical registering matrix was designed and a series of recommendations 
are presented for use by doctoral programs and graduate programs generally. 

Keywords: doctoral programs; performance measurement; completion rates; time-to- 
degree; graduate statistics 

El desempeno en el nivel doctoral de educacion en cifras: Ausencia de informacion 
y sugerencias para su produccion 

Resumen: Desde mediados de los 90 en la Argentina y en America Latina, se ha producido un 
crecimiento abrupto del nivel posgrado. Sin embargo, las tasas de graduation doctoral (y de 
posgrado en general) estan lejos de ser satisfactorias. Desde el sentido comun, “se sabe” que la 
graduation de los posgrados es baja, y que esto es mas asi en Ciencias Sociales y Humanidades (que 
mas adelante denominaremos “Ciencias Blandas”) que en las Exactas y Naturales (que mas adelante 
denominaremos “Ciencias Duras”), aunque no existe informacion completa, valida y confiable. Para 
enfrentar este vacio llevamos a cabo una investigation con dos objetivos: medir el desempeno 
(eficiencia o productividad) de 18 programas doctorales en Ciencias Exactas y Naturales y en 
Ciencias Sociales y Humanidades de unidades academicas con sede en Buenos Aires y el Area 
Metropolitana; y disenar un procedimiento para producir informacion valida y confiable sobre el 
desempeno. Este se definio en terminos de dos dimensiones: tasas de graduation (en tiempo 
reglamentario y tiempo extra reglamentario) y tiempo hasta la graduation promedio por cohorte. 
Para ello registramos la trayectoria academica de los individuos de cada cohorte desde su inscription 
hasta su graduation o desertion. Analizamos la informacion por cada programa y para seis cohortes 
agrupadas (2001-2006), tambien por cada programa. Nuestros resultados constataron con datos 
confiables y validos que el desempeno en las Ciencias Exactas y Naturales es mejor que en las 
Ciencias Sociales y Humanidades; y ademas, disenamos una matriz de registro de estadisticas de 
posgrado y una serie de recomendaciones para que los programas doctorales y de posgrado en 
general puedan recoger sus propias estadisticas. 

Palabras-clave: programas doctorales; medicion del desempeno; tasas de graduation; tiempo 
promedio a la graduation; estadisticas de posgrado 

O desempenho na educa§ao em nivel de doutorado em numeros: A falta de 
informa^oes e sugestoes para a sua produ§ao 

Resumo: Desde meados dos anos 90 na Argentina e na America Latina, e produziu um crescimento 
abmpto da nivel de pos-gradua^ao. No entanto, as taxas de graduayao doutorada (e de pos- 
gradua^ao em geral) estao longe de ser satisfatoria. Do senso comum, “conhecido” que o ranking 
dos programas de pos-gradua^ao e baixo, e que isto e tao mais em ciencias sociais e Humanidades 
(que mais tarde serao chamadas de “Ciencias Soft”) na exata e natural (que mais tarde sera chamado 
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de “Ciencias duras”), embora nao haja nenhuma informa^ao completa, valida e confiavel. Para 
enfrentar este vazio transportar para fora de uma pesquisa com dois objetivos: medir o desempenho 
(eficiencia ou produtividade) dos 18 programas de doutorado em ciencias exatas e naturais e em 
ciencias sociais e humanas das unidades academicas, com sede em Buenos Aires e a area 
metropolitana; e criar um procedimento para produzir informa^Ses validas e confiaveis de 
desempenho. Isto foi definido em termos de duas dimensoes: taxas de gradua^ao (no tempo 
regulamentar e prolongamento) e coorte de tempo-para-media da gradua^ao. Entao, nos gravamos a 
carreira academica dos individuos em cada coorte de sua inscri^ao para gradua^ao ou abandono. 
Analisamos as informa^oes para cada programa e seis coortes agrupados (2001-2006), tambem por 
cada programa. Nossos resultados confirmados com dados validos e confiaveis sobre o desempenho 
de exatas e ciencias naturais e melhor do que nas ciencias sociais e Humanidades; e alem disso, nos 
projetamos uma matriz de registro de pos-gradua^ao estatisticas e uma serie de recomenda^oes para 
que os programas de doutorado e pos-gradua^ao em geral podem coletar suas proprias estatisticas. 
Palavras-chave: programas de doutorado; medi^ao de desempenho; taxas de gradua^ao; tempo 
medio de gradua^ao; pos-gradua^ao estatisticas 

Introduction 

Since the mid-1990s, Argentina experienced an abmpt growth in the supply of doctoral 
programs and in enrollment demand. Since the 1970s, the “knowledge society” has become relevant, 
as the production and circulation of knowledge has acquired prominence in a context where 
knowledge and the economy became closely linked (Castells, 1997). In Argentina, the Higher 
Education Law, passed in 1995, validated an educational model that emphasized the production and 
transmission of knowledge, as well the exponential growth of graduate programs. Apparendy (if the 
statistics are trusted, although below we will question their reliability), the total supply of graduate 
courses more than tripled between 1994 and 2014, rising from 793 to 2098 (Fliguer & Davila, 2010; 
SPU, 2015). This movement was differential: graduate Specializations grew the most (from 301 to 
964), followed by Masters’ programs (from 246 to 742), whereas growth at the Doctoral level, was 
the lowest (246 to 392). The corresponding three types of graduates followed similar patterns with 
an abrupt rise between 1994 and 2002. Subsequently, these increases slowed, but the tendency was 
always increasing. 

Similar growth in graduate programs took place in different world contexts. For example, 
Bowen & Rudenstine (1992), Ehrenberg et al. (2007, 2010) and Lovitts (2005, 2008,2011) and Kiley 
note this phenomenon in the United States; Jiranek (2010), Halse and Mowbray (2009) observe it in 
Australia; and De Miguel, Sarabia Heydrich & Amirah (2004) also find similar trends in several 
European countries. For Latin American, Tunon (2012) cites studies tracking the same trends in 
Venezuela, Colombia Mexico, and Chile. Moreover, they all agree that a large percentage of doctoral 
students fail to finish their programs, suffer frequent intermptions in their studies, and that the track 
duration is very long for those students who do graduate. 

Growth in the supply of graduate programs has been accompanied by strong pressure from 
government agencies for accountability and performativity. In Argentina this trend is represented by 
the National Commission for Higher Education Evaluation and Accreditation (CONEAU, 

Comision Nacional de Evaluation y Acreditacion Universitaria) and the National Ministry of 
Science, Technology & Productive Innovation (MinCyT, Ministerio de Ciencia, Tecnologia e 
Innova cion Productiva de la Nation). However, doctoral completion rates are far from being 
satisfactory in Argentina and around the world. Furthermore, these rates differ by discipline: they are 
higher in STEM disciplines than in the Social Sciences and Humanities. 
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In terms of completion rates, Zainal Abiddin & Ismail (2011) cite Elgar's study about 
Canada. In Canada only 45% of doctoral students in the Arts and Humanities complete their 
program, while in Life Sciences and the Sciences (understood as STEM disciplines) the percentage 
of completion rises to 70% and 60%, respectively (2011). Wright and Cochrane present a similar 
scenario for the United Kingdom, where 51% of doctoral students in the Arts and Humanities 
complete their program, while in the Sciences (also understood as STEM disciplines) 64% of 
doctoral students reach this objective (2000 cited in Zainal Abiddin & Ismail, 2011). In the United 
States, graduation rates vary between 30% and 50% in different fields within the Humanities; 
between 50% and 70% in STEM disciplines, (Lovitts, 2001); and between 35% and 60% in the 
Social Sciences (Gardner, 2013). Gardner cites Lovitts (2001), Golde (2005, 2007) and Millett & 
Nettle (2006) to note that during the 1990s completion rates in the US varied between 11% and 68% 
depending on the field. For Australia, Kiley (2011) observed that only 50% of the doctoral students 
who began their studies between 2005 and 2011 had completed their programs. Jiranek (2010), who 
also studied Australia, found completion rates in the Natural Sciences to stand between 60% and 
70%, and from 49% to 55% in the Social and Human Sciences. De Miguel, Sarabia Heydrich & 
Amirah (2004) found a similar scenario in Spain: in the 1990s, Experimental, Social and Human 
Sciences showed completion rates of 52%, 23% and 14% respectively. As this brief review shows, 
figures vary across disciplines and across countries, but there are two common trends: completion 
rates do not reach 80% and they are consistendy higher in Natural Sciences (the STEM disciplines) 
than in Social Sciences and Humanities. 

We must stress that the availability of complete, valid and reliable information at the level of 
higher education is very low. In this context "it is known" that attrition is high and that attrition 
rates are higher in the Social Sciences and Humanities than in the Natural Sciences. In addition, “it is 
known” that attrition rates are higher at the dissertation-production stage than at the stage of 
coursework (giving rise to the “All But Dissertation”/ABD concept). These claims are based on 
abundant observations in few cases, which are collected with little or no systematic methodology. 

The absence of valid and reliable statistics in part is due to characteristics of the graduate 
educational level. Unlike the primary and secondary levels, higher education at the graduate level is 
not obligatory, has no strict terms of completion, abounds in extensions and leaves, and allows 
"unregistered migrations" between programs within and/or between academic units. For example, 
often, there is no single starting date that makes it possible to identify cohorts and calculate the 
stock of students at any given moment. The absence of valid and reliable statistics also reflects the 
great heterogeneity within postgraduate programs, particularly in doctoral programs: these programs 
may have a structured, semi-structured, or customized format; they may or may not have a cohort 
regime (same date initiation); and dissertation work may be simultaneous to coursework or 
subsequent to completion of coursework, etc. Furthermore, many academic departments place little 
importance upon statistical information as a necessary tool for tracking and monitoring postgraduate 
programs and for policy evaluation. On the other hand, accrediting agencies value statistical 
information, as they emphasize accountability. In Argentina, the CONEAU is interested in this 
evaluation, something which seems to be common throughout the world, at least for the countries 
for which we have information. 

In Argentina, as we have already noted, since the 1990s there has been a strong focus on 
higher education evaluation following the guidelines of international agencies (Krostch, 2002). As a 
result major changes were introduced in two dimensions in universities: on the one hand, in general 
funding, teaching and research and, on the other, in increasing accountability (Escotet & Aiello, 

2010; Fernandez Lamarra, 2003). 
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As regards the first dimension, some of the policies put into practice were: the creation of 
the Higher Education Quality Improvement Fund (FOMEC, Fondo para el Mejoramiento de la 
Calidad Universitaria) ; and the Incentives Program for Research-Professors ; an increase in the 
number of doctoral scholarships offered by the National Council for Scientific and Technical 
Investigations (CONICET, Consejo Nacional de Investigaciones Cientfficas y Tecnicas) and the 
National Agency for the Scientific and Technical Promotion (ANPCyT, Agenda Nacional para la 
Promotion de la Ciencia y la Tecnologfa) . 

As for the second dimension, the creation in 1993 of the Higher Education Policy Secretariat 
(SPU, Secretarfa de Politicas Universitarias), within the Ministry of National Education, deserves 
special attention. The policy cluster promoted under the wing of the SPU constitutes the new system 
of higher education. This system pursues a strong emphasis on the development and evaluation of 
graduate programs' quality and is associated with the creation of a Higher Education Information 
System (SIU, Sistema de Information Universitaria) to optimize performance and efficiency. In 
response, the SPU should systematize statistical information provided by universities and publish 
yearbooks or graduate guide-books for researchers and civil society. 

The SPU's tasks are combined with the assessment and accreditation system established by 
the CONEAU in 1995 to evaluate operating and proposed universities, and to regulated 
undergraduate and graduate programs through their accreditation. This secretariat on higher 
education pursues three objectives: “to promote the consolidation and evaluation of the graduate 
system following internationally recognized criteria of excellence; to promote the training of highly 
qualified human resources, for both academic teaching and research activities, as well as professional 
specialization; and to offer society reliable information about graduate educational quality, so to 
expand the capacity of prospective student's to make choices” (Marquis, 2009, 50). The CONEAU, 
which is regulated by the Higher Education Law (LES, Ley de Education Superior), is structured to 
give the state a major role in evaluating higher education and has evaluation as a central criteria for 
its policy (Krotsch, 2002). 

We maintain, as did Marquis (2009, 39), that “there is little consistent and reliable 
information about postgraduate programs for Argentina” despite this cluster of policies. This 
comment echoes an observation made for another context: “UNESCO, OECD, and even 
EUROSTAT's global statistics do not give reliable data on the doctoral level [...] There are therefore 
no comparable international statistics.” (De Miguel et al. 2004, p.148). 

The available data does not allow us, among other things, to measure retention and student 
completion in both undergraduate and graduate programs, nor to pursue systematic studies about 
causes of attrition. 

An Attempt at Valid Measurement of Doctoral Program Efficiency 

We sought to measure doctoral program efficiency in various disciplines in terms of 
completion rates and average time-to-degree by cohort using data that we collected. To 
contextualize, we note that this work was part of a research program aimed at producing knowledge 
about how researchers are trained. This research included the following aspects: institutional and 
pedagogical factors, “learning environments”, thesis strategies for students, and training provided by 
dissertation supervisors and theses workshops/seminar teachers. We focused on doctoral programs 
because although possessing a doctorate is not necessarily a requirement for being a researcher, 
generally having a doctoral degree is the entry requirement for passage into the academic research 
community. 

In other words, we carried out research with two objectives: (1) to evaluate the performance 
(efficiency or productivity) of 18 doctoral programs in the Social Sciences and Humanities and in the 
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Natural Sciences from in different academic departments from five universities (these universities, 
located in Buenos Aires and its Metropolitan Area, included public and private institutions); and (2) 
to design a methodology to produce valid and reliable information on the performance (efficiency or 
productivity) of graduate programs to record data for their monitoring. 

We defined program performance in terms of two dimensions: (i) completion and (ii) time- 
to- degree. We measured completion (i) with two indicators: (i.a) overall completion rates (OCR) 
and (i.b) regular (set by each doctoral program) time completion rates (RTCR). The second 
dimension, time-to-degree (ii), was measured in terms of average time-to-degree (TTD). Since 
programs are differently organized, we defined a cohort as the group of students who are admitted 
in a calendar year regardless students' different admission dates within the same year. In 
consequence, this enabled calculations for each cohort. 

After examining the quality of the information produced by the SPU and its data loading 
system (SIU, Sistema de Information Universitaria), we decided to design a methodology to measure 
efficiency. Universities are required to collect data from undergraduate and postgraduate programs 
for SIU and send them to the SPU to be processed and published in statistical Yearbooks. 
Particularly, a first Yearbook exclusively on graduate programs (masters and doctoral degrees) was 
published in 1997 within the Higher Education's Information System Improvement Program 
(PMSIU, Programa de Mejoramiento del Sistema de Information Universitaria). Although PMSIU 
ran between 1997 and 2004, it did not produce information until 2006. After eight years of inactivity, 
statistics were published annually from 2006 to 2012. 

In addition to the interruptions which do not provide statistical series, there are no 
mechanisms that oblige universities to send information to the SPU nor to rigorously load data into 
the SIU. As a consequence, certain yearbooks clarified that certain universities had been excluded 
from the publication for diverse reasons, such as: (i) failure to regularly collect this information at 
the graduate level; (ii) failure to accredit their graduate offerings; and (iii) failure to respond to SPU's 
requests for information. Both in Estadisticas 1997 (Statistics, 1997) and in Anuario 2006 de 
Estadfsticas Universitarias (Higher Education 2006 Yearbook), it is noted that “data collection and 
its consolidation meant an effort for universities because [...] there is no precise, in quantitative 
terms, knowledge about the postgraduate level, since such information is not traditionally collected 
by statistics departments in institutions of higher education.” The Anuario 2008 de Estadisticas 
Universitarias (Higher Education Statistics 2008 Yearbook) noted that “[publishers] cannot present 
historical series due to the lack of information that might rigorously illustrate postgraduate 
enrollment growth, in recent years. [They] hope to achieve greater coverage and thus be able to have 
reliable historical information in time.” (SPU, 2008, 125). Similarly, the Anuario 2010 de Estadfsticas 
Universitarias (SPU, 2010) clarifies that "even though measuring the students population [referring 
to graduate students] has been improving data coverage, there are institutions that still supply partial 
information” (p. 7). 

Other authors have been already warned about these points. Marquis (2009) sustained that 
“[SPU] information was built on data provided by universities that did not necessarily use the same 
criteria and procedures to collect their data, and there were time periods for which they did not 
report information or they did so in an incomplete way” (p. 39). Garcia de Fanelli (2004) and 
Kisilevsky (1999) noted the asymmetry in information between universities and the government: the 
universities have more information than the government agencies. Regarding this problem, the SPU 
itself in several Yearbooks has warned that “the information in this chapter may be under 
representative”. The 2006 Yearbook said that: “today, 39% of the national universities that offer 
postgraduate degrees have not reported the number of students for at least one year during the 
period 1997-2005, during which there is no information for periods of three, four, and more years”. 
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This lack of data makes it impossible to build a valid view of reality; however, the SPU considers 
that its reports provide an interesting database, even with gaps that require improvement. 

Many researchers have used information drawn from these yearbooks to analyze the 
evolution of graduate programs without warning readers about such shortcomings. Thus, several 
works using the same SPU statistics agree on the increase in supply of graduate education, but they 
do not agree on the absolute numbers. On this subject, Tunon (2012) referred to Barsky's work 
(1999) and noted that “there are data in the same study that show a difference in the total supply of 
almost 100 postgraduate programs in absolute numbers, representing a 5% variation in percentage 
terms” (p. 30). 

In addition to the problems mentioned above, the SPU does not produce adequate 
information about the efficiency of the system. The doctoral completion/attrition rates that are 
calculated and published are not really indicators of completion and attrition since they represent the 
relationship between the number of new enrolled students and the number of completing-students 
in a calendar year. In other words, it compares the cumulative stock of students (enrolling students) 
with accumulated completing-students stock. This implies that “efficiency” is not measured in terms 
of the ratio between input and output. To measure the latter, we must know the number of students 
who graduated after a period of time out of those who were enrolled in the same year; i.e. the 
number of completing-students of the same cohort. 

In sum, the SPU occasionally provides information, of questionable validity and reliability, 
with faulty coverage, that is not comparable over time. This situation explains why we were forced 
to produce our own data and, by doing so, design specific processing techniques. 

As soon as we started our research, it became clear that we were going to be unable to 
analyze the system as a whole because the population of Ph.D. students (and the population of 
students in higher education in general) is an ‘open’ population in which (unlike the primary and 
secondary levels) permanence within the Argentina's graduate system is not recorded. For example, 
if a student does not complete a program, it is impossible to know if he/she migrated to another 
program or if he/she dropped out of the system (this is possible for the primary or secondary level 
in since “migration” between schools is recorded within this country). This means that what we 
evaluated was each doctoral program’s efficiency; we did not study completion or attrition 
conditions of each individual that went through a doctoral program. Global doctoral efficiency, 
understood as completion of studies (which is accurate) and not attrition (which is inaccurate), can 
only be assessed by cumulating information from all program. 

The construction of such statistics was a complex task, especially for doctoral programs, due 
to the lack of records registering permanence, as well as for other reasons, such as the absence of 
shared definitions regarding curricular and organizational structure. Even though the regulations of 
almost every doctoral program define a time limit for completion, these time periods are not 
identical, not nor are there homogeneous policies regarding extensions, leaves, and readmissions. 
Nevertheless, data was available from the universities and we used it, but only after evaluating and 
cleaning it before processing. It should be stressed that we sought to evaluate the efficiency, not the 
quality of doctoral programs. 


Methodological Aspects 

We selected the 18 doctoral programs based on theoretical and pragmatic criteria. The first 
led us to include different disciplinary fields; the second led us to take into account the accessibility 
of the data. There is an abundant literature on the theoretical criteria. Some studies are stronger than 
others in terms of their empirical basis, but all studies agree that doctoral completion rates for the 
Natural Sciences are higher than those in the Social and Human Sciences. There are multiple reasons 
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for this finding. Perhaps, “learning environments” and working “practice communities” are the main 
reasons, as well as differential access to scholarships among diverse disciplinary fields. If comparison 
in efficiency between fields revealed systematic differences, it would allow us to speculate about the 
factors involved in such events. Moreover, it would take us closer to our ultimate goal, which is to 
design strategies (educational, institutional, organizational, curricular) aimed at improving the 
performance of doctoral student training and to lower attrition. 

We collected data for each individual student throughout his/her academic career (identified 
by a numeric code in replacement of first and last name) and then we added them by cohort or 
initiating year into the program in order to calculate completion rates and average TTD. Thus, the 
unit of measurement was the student and the unit of analysis was the cohort of students admitted in 
the same year. Hence, we designed a data collecting matrix using each doctoral students’ academic 
tracks and milestones (see Figure 1). As it was said, our research was intended to measure not only 
efficiency, but also to identify milestones at which points program attrition was higher. Paying 
special attention to these points might have offered an enriching perspective to design efficiency 
improvement strategies. Unfortunately, we could only identify such milestones in only a few of the 
18 doctoral programs because of a lack of original records. In consequence, these aspects were not 
included in this paper. 


Figure 1 

Data collecting fnatrixfor doctoralprograms’ academic tracks. * 
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Source: Tunon, 2012. p. 59. * Records for courses approval were added. Matrix based on Wainerman and Di 
Virgilio’s proposal (2009) for a major program’s project. 


In most programs investigated this approach led us to an input-output study; thus, a “black 
box” study, by considering the number of incoming (admission date) and the number of graduates 
(dissertation date). If intermediate milestones information had been accessible, it would had been 
possible to go into the “black box.” 

As we said, we studied 18 doctoral programs at universities from the public and private sector 
located in the Buenos Aires’s Metropolitan Area (AMBA, Area Metropolitana de Buenos Aires) in 
Soft Sciences - seven programs - and Hard Sciences - 11 programs. We used Becher’s classical works 
(1989, 1993), Becher & Kogan (1992), Light (1974), Biglan (1973), Lodhal and Gordon (1972), Ladd 
and Lipset (1976) and Kolb (1981) to select diverse disciplinary fields. Becher himself took 
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references from these works’ models and elaborated a new classification, which is adopted in this 
research. This classification is organized in poles: hard-soft and pure-applied. However, we only 
took hard-soft pole. 

Hard disciplines are sometimes described as cumulative: they pursue atomistic knowledge 
(tree-shaped or pyramid-shaped); generally, they care about universal topics, quantities and 
simplification, and specific matters; and results are associated with discoveries and explanation. In 
addition, Hard disciplines are moved by pragmatic purposes; they care about the domain of the 
physical environment; often, their results are products and techniques. Soft disciplines are repetitive 
and holistic; they care about specific topics and quality; and results are associated with 
comprehension and understanding. Moreover, Soft disciplines are also functional and utilitarian. 
They care about professional practices, and results are based on protocols and procedures (Becher, 
1993). 

In order to facilitate interdisciplinary comparison, we added six starting cohorts in each 
program between 2001 and 2006. The year 2001 was selected as a starting point on the basis of the 
creation date of the most recent program. Given that there are regulations that allow for up to eight 
years for completion (with the inclusion of two one-year extensions) we selected 2006 as an end 
point to allow cohorts from all 18 programs to achieve completion at the time that we collected data 
(2014). Programs varied in regards to management sectors, i.e. whether they were private or public; 
age as measured from year of creation; size in terms of enrollments; curricular stmcture, i.e. whether 
they involved custom designs, semi-structured, or stmctured formats; the modality of education, i.e. 
face-to-face, blended, or virtual; the nature of the dissertation production period — alongside 
coursework or subsequent to completion of courses; and the amount of time allotted for dissertation 
completion. These program characteristics are summarized in Table 1. 

Before collecting the data, we acquainted ourselves with each program's regulations, and 
curricular and organizational stmcture. To do this, we analyzed documents and interviewed program 
directors or academic assistants. As shown in Table 1, all of the “hard” programs are run in public 
universities. Whereas, “soft” programs are found within public and private universities. Most 
programs are customized; all of them are face-to-face; most of them separate the stage of 
coursework from the stage of thesis preparation; and regulations for completion specify periods that 
vary between 4 and 8 years, including permissible time extensions. 
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Table 1 


Doctoralprogram characteristics for 2001 -2006 cohorts 
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After 

courses 

8 

Inorganic 

chemistry, 

physical 

chemistry and 

analytical 

chemistry 

Public 

1984 

60 

Custom 

Face to 
face 

After 

courses 

8 

Biological 

chemistry 

Public 

1897 

172 

Custom 

Face to 
face 

After 

courses 

8 

Physical 
sciences 1 

Public 

1898 

128 

Custom 

Face to 
face 

After 

courses 

8 

Biological 

sciences 

Public 

1897 

623 

Custom 

Face to 
face 

After 

courses 

8 

Organic 

chemistry 

Public 

1987 

40 

Custom 

Face to 
face 

After 

courses 

8 

Mathematical 

sciences 

Public 

1953 

55 

Custom 

Face to 
face 

After 

courses 

8 

Atmospheric 
and ocean 
sciences 

Public 

1973 

20 

Custom 

Face to 
face 

After 

courses 

8 

Geological 

sciences 

Public 

1897 

68 

Custom 

Face to 
face 

After 

courses 

8 

Computer 

sciences 

Public 

1989 

64 

Custom 

Face to 
face 

After 

courses 

8 
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Table 1. (cont.'2) 

Doctoralprogram characteristics for 2001 -2006 cohorts 


Disciplinary 

fields 

Management 

sector 

Age 

(creation 

year) 

Enrolled 

Students 

2001- 

2006 

Curricular 

stmcture 

Modality 

Timing of 
thesis 

preparation 

Regular 

completion 

time 

(years) 

SOFT SCIENCES 

Social 
sciences 2 

Private 

2002 

157 

Semi 

structured 

Face to 
face 

Along 

with 

courses 

5 

Social 
sciences 1 

Public 

1999 

459 

Custom. 

Face to 
face 

After 

courses 

8 

Education 

sciences 

Private 

2001 

25 

Semi 

stmctured 

Face to 
face 

Along 

courses 

4 

History 

Private 

1999 

18 

Semi 

stmctured 

Face to 
face 

Along 

courses 

4 

Political 

sciences 

Private 

1966 

39 

Semi 

stmctured 

Face to 
face 

After 

courses 

None 

Sociology 

Private 

1989 

17 

Stmctured 

Face to 
face 

After 

courses 

8 

International 

relations 

Private 

1966 

58 

Semi 

stmctured 

Face to 
face 

After 

courses 

None 


Note: There are two Physical Sciences programmes (1 & 2) and two Social Sciences programmes because they 
take place in different universities. 


We produced three indicators for the completion dimension: overall completion rate (OCR), 
regular time completion rate (RTCR) and extra time completion rate (ETCR). The first is defined as 
the ratio between the number of graduates and the number of admitted students over a period of 
time; RTCR measures the number of graduates that completed the program in regular time; while 
ETCR is a measure of the number of graduates that completed the program after regular deadlines. 
Therefore, OCR is equal to the sum of RTCR and ETCR. 

We developed only one indicator for the time-to-degree dimension: the global average time- 
to-degree. This is the average time taken by all graduates within a set who began at the same date 
(year) to complete their PhDs. (i.e. the difference between the ‘dissertation date’ and the ‘admission 
date’). 

As regards these indicators, it is important to note that the population studied by these 
completion rates - OCR, RTCR and ETCR - and the average time-to-degree always encompasses 
the set of admitted students in a delimited period of time; in this case, all individuals admitted 
between 2001 and 2006. In other words, it is the temporal definition of admission that defines the 
population studied. This clarification has substantive implications because it differentiates these 
indicators from those used by the SPU. As mentioned before, this organization works with 
cumulated graduates stocks per year. 


Results 
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As already stated, our results are based on a study of 18 doctoral programs: 11 from fields in 
the “Hard sciences” and seven from fields in the “Soft Sciences” (Becher, 1993) (see Table 2). 

OCRs from these 18 doctoral programs vary between 9% and 100%, with systematic 
differences between disciplines. OCR rates among the Hard sciences vary between 45% and 100%, 
while these rates for the Soft sciences vary between 9% and 100% (see Table 2). Within the first 
group, Physical science 2 has the highest OCR, while Computer science has the lowest (45%). In the 
second group, Social science 1 and 2 (55% and 57%, respectively) have the highest OCR and 
International Relations has the lowest (9%). 

Note that if Computer science were excluded from the Hard Sciences group, the Soft Sciences 
program with the highest OCR, would have a rate lower than the program with the lowest OCR in 
the Hard Sciences. This initial reading of the data clearly points to differences by disciplinary fields 
in regards to completion rates within efficiency levels. 

In the Hard or Natural sciences the TTD dimension reveals a pattern similar to that of the 
OCR: Physical science 2 has the lowest average time to degree (4.4 years) and Computer science has 
the highest (5.58 years). This pattern is not repeated in the Soft sciences: the program with the 
lowest TTD, Sociology (4.41 years), does not have the highest OCR (Social sciences 2). In addition, 
the lowest TTD program (Sociology, 4.41 years) is far from having the highest OCR (Social science 
1 and 2, 55% and 57%, respectively). 

It is worth noting that even though TTDs for the Hard sciences are lower than those of the 
Soft sciences, the differences between these times are smaller than the differences among OCRs. 
Perhaps, RTCRs combine indicators from both efficiency dimensions because they consider time 
deadlines established by academic departments. 

We can detect two patterns of behavior for the Hard Sciences with these measures. First, 
RTCRs correlate identically with OCRs. Thus, ordering programs by highest-to-lowest OCR 
reiterates the program ordering by RTCR. Secondly, differences between both rates are very small, 
5% points at most. This means not only that a greater proportion of students graduate in doctoral 
programs in the Hard sciences than in Soft science programs, but that students complete their 
degrees in most programs by respecting regular deadlines. Only a minimum fraction of students 
complete doctoral programs outside the time limits established by academic departments. 

Table 2 


Doctoralprograms by Becher’s, enrollment, OCRs, RTCR, average time-to-degree, OCRs range of cohorts 2001 -2006, 
by doctoralprogram 


Disciplinary fields 

Enrollment 

(N) 

2001-2006 

Overall 

graduation 

rate 

Regular 

Time 

Completion 

Rate 

Average 

time-to- 

degree 

(years) 

OCRs 

range 

within 

cohorts 

HARD SCIENCES 

Physical sciences 2 

13 

100% 

92% 

4.40 

100% 

Industrial chemistry 

34 

85% 

85% 

5.33 

50% - 100% 

Inorganic chemistry. 

physical chemistry 
and analytical 
chemistry 

60 

83% 

82% 

5.00 

60% - 100% 


Table 2. (cont.'d) 
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Doctoralprograms by Berber’s, enrollment, OCRs, RTCR, average time-to-degree, OCRs range of cohorts 2001 -2006, 


by doctoralprogram 


Disciplinary fields 

Enrollment 

(N) 

2001-2006 

Overall 

graduation 

rate 

Regular 

Time 

Completion 

Rate 

Average 

time-to- 

degree 

(years) 

OCRs 

range 

within 

cohorts 

HARD SCIENCES 

Biological 

Chemistry 

172 

83% 

81% 

4.85 

68% - 90% 

Physical science 1 

128 

80% 

76% 

5.01 

67% - 95% 

Biological science 

623 

78% 

75% 

4.82 

71% - 87% 

Organic chemistry 

40 

78% 

75% 

5.32 

50% - 100% 

Mathematical 

science 

55 

73% 

69% 

5.01 

57% - 85% 

Atmospheric and 
ocean Science 

20 

70% 

65% 

5.09 

33% - 100% 

Geological Science 

68 

66% 

63% 

5.04 

29% - 86% 

Computer science 

64 

45% 

41% 

5.53 

15%-70% 

SOFT SCIENCES 

Social science 2 

157 

57% 

13% 

5.84 

39% - 80% 

Social science 1 

459 

55% 

47% 

5.55 

30% - 78% 

Education science 

25 

44% 

4% 

6.07 

0%-100% 

History 

18 

28% 

0% 

5.47 

0%-100% 

Political science 

39 

21% 

* 

4.76 

0% - 40% 

Sociology 

17 

24% 

18% 

4.41 

0% - 40% 

International 

relations 

58 

9% 

* 

5.48 

0% - 25% 


* Doctoral programs without regular time deadlines. 


The situation in the Soft sciences is very different. Differences between program OCRs and 
RTCRs are very broad. This difference implies that only a minority of students graduate by regular 
time deadlines. This pattern is reflected in another characteristic: the highest-to-lowest ordering of 
programs by OCR and by RTCR does not coincide. 

The efficiency levels found here reproduce similar findings in other contexts such as the 
United States (Bowen & Rudenstine, 1992; Ehrenberg, Zuckerman, & Groen, 2010; Ferrer de 
Valero, 2001; Gardner, 2008, 2009, 2010; Golde, 2005, 2007; Kim & Otts, 2010; Main, 2014; Millett, 
2006; Vassil & Solvak, 2012): doctoral programs in the Hard sciences (Natural Sciences in OECD 
terms) have higher efficiency levels than programs in the Soft sciences (Social sciences and 
Humanities in OECD terms). In addition, the behavior of the indicators is more homogeneous in 
than Hard sciences than in the Soft. 

We hypothesize that differences in efficiency levels between disciplinary fields are related to 
several factors discussed in Becher’s studies about academic tribes (Becher, 1989; 1993; Becher & 
Kogan, 1992): initiation modes, forms of social interaction, the nature of knowledge in each 
discipline, and access to financing. 
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So far we have presented an efficiency evaluation of doctoral programs, our first objective. 

As for the second objective, the design of procedures to carry out an efficiency evaluation, we list 
several recommendations that emerged from our experience and discuss challenges. 

The first and foremost recommendation is to gain information access. Even though some 
academic departments have records and are willing to show them subject to individual data 
confidentiality restrictions, other departments do not maintain valid and complete records or they do 
not process them. Some departments are very reluctant to provide such data for research either 
because they are unaware of its importance, fear that doing so will dismpt their day-to-day 
operations or, probably, fear discovery and publicity regarding the low efficiency of their doctoral 
programs. 

Second, attrition evaluation is only possible for each individual program, not for the doctoral 
system as a whole. This limitation is due to the absence of information with which to determine 
whether a doctoral student who does not complete a program is dropping out or is migrating to 
another doctoral program within the system. 

Third, since doctoral programs do not always record when students drop out, to study 
attrition it is necessary to trace in each cohort each students’ academic track from date of admission 
and then detect when the student disappears from the records. 

Fourth, to develop a valid calculation of overall performance and performance at different 
stages, it is necessary to have information about each doctoral programs’ regulatory, organizational, 
and curricular characteristics (structured, semi-structured or custom), dissertation preparation 
periods (after courses or simultaneously), academic track milestones, and the associated regular time 
deadlines, among others. 

Fifth, there may have been changes in the course of a program’s history, for example, in 
deadlines for completion or in the requirements at each intermediate curricular stage. Therefore, in 
such cases it may be necessary to correct the completion-attrition criteria used when studying 
program efficiency in historical series. 

Sixth, after verifying the existence of data, its location, and the manner of its recording, the 
researcher should use the most primary and primary and disaggregated sources so as not to lose 
information. 

Seventh, it is necessary to investigate how regulations are put into practice. It is very 
important to understand mechanisms used to “skip” program regulations; for example, when a 
person exceeds deadlines and fails to complete their dissertations, why isn't the person considered a 
drop out? 

Eighth, we recommend distinguishing between two completion times: regular-time and 
overall completion time. When doctoral students take longer than they “should”, the system is not 
failing at getting the student to complete the program, but the system is performing “less 
efficiently”. 

Ninth, counting months and counting years produce different results when calculating TTD 
and completion rates. In addition, if years are counted the results differ according to whether one 
counts the year that is beginning or counts only completed years. Counting months is a more 
accurate way to measure students’ academic tracks. 

Tenth, when there is a discrepancy between the date specified in a database and the date 
found in an official resolution for the same stage, we recommend using the oldest date (usually in 
the database) as resolutions are often delayed for years. 

These are several methodological challenges that we faced in our study. Although there are 
many challenges, we stress the importance of measuring the efficiency of doctoral programs. Such 
measurements provide a powerful tool for evaluating performance. Institutions need to increase 
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their awareness regarding the importance of measuring efficiency, in order to understand their own 
strengths and weaknesses, and to improve “their service” and performance. These tasks require that 
qualified personnel collect relevant data. 

Doctoral programs, regardless of their size, should record dates of: admission, registration, 
advisor appointment, dissertation supervisor appointment, and project proposal approval, 
completion of the dissertation, and dissertation defense. These records should also include 
information related to the student's coursework, including: lists of the courses in which the student 
enrolled, enrollment dates, approval dates, and the number of credits granted. Once such data is 
recorded, they must be processed for the bodies supervising the program, and they should be made 
available for public access in the interest of complying with desirable transparency requirements. 

Discussion 

We observed that completions rates (OCR, RTCR and ETCR) and time-to-degree for the 
Hard science disciplinary fields were better than those rates and times of the Soft sciences by 
studying 18 doctoral programs in Argentine universities. Observation of these differences do not 
lead us to rank one disciplinary field over another, nor to recommend transferring one field's 
working models to another, because generally disciplines are organized around very different and 
unavoidable epistemic differences. However, we are aware that the Soft sciences are characterized by 
higher attrition and by longer completion times. These problems deserve an improvement trial. 

Although after months of lobbying we were welcomed in several academic departments, our 
experience revealed a series of problems with data production at the doctoral level: (1) in some 
departments we discovered an absence of valid and reliable information; (2) authorities were often 
unaware of the need to record this information for departmental management; and (3) as a 
consequence, such information never made it to the national agency responsible for receiving, 
processing, and publishing this information for society’s knowledge. On the other hand, our 
experience revealed that with persistence, wit, and tolerance researchers can overcome institutional 
resistance and gain access to the information needed for evaluating the efficiency of doctoral 
programs. 

We insist that studies like ours are needed to provide valid and accurate baseline data. We 
need such data before we can attempt to evaluate factors that might be associated with attrition, 
identify those teaching strategies that are conducive to completion, as well as to identify at which 
stages of the academic track curricular and pedagogical changes are needed to improve doctoral 
efficiency. 

We agree with Moler (2008), the SPU’s Research and Statistical Information coordinator, 
when she warned that "educational statistics are an essential tool to generate effective knowledge 
about the reality of higher education [...] since it is necessary to obtain a correct evaluation of the 
impact, progress, and setbacks within the national university system (p. 263.) We also recognize that 
the “construction of information systems in all organizations requires profound cultural changes. If 
we do not acknowledge this point [...], we will not understand the situations that directly affecting 
the compilation of information” (Moler, 2008, p. 265). 

Finally, we want to stress that we study efficiency levels, their determinants, and associated 
variables not only with the goal of quantifying the information. Rather we are also interested in 
studying the functioning of doctoral programs because we are committed to improvements that 
contribute to increasing the number of scientific researchers in Argentina in the context of a 
“knowledge society” and to assuring the efficient use of always scarce financial resources. 
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In addition, we sustain that once data about completion, attrition and duration are produced, these 
might enable the study of factors that hinder and facilitate degree completion. Furthermore, it would 
allow to design curricular or pedagogical mechanisms to overcome greater attrition stages in 
academic tracks in diverse fields. 
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